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significant positive correlation was found between students' 
cumulative writing experience and the quality of their writing; (3) 
the study of grammar was of little or no benefit to the improvement 
of writing; (4) though no significant correlation existed, an 
emergent pattern suggested that WAC instruction may improve students' 
mastery of course objectives; and (5) students who had more writing 
experience in community colleges had a more positive attitude toward 
writing as well as better comprehension of subject material. Though 
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Faculty interviews, student and faculty surveys, holistic ratings 
of student compositions, and trait ratings of essay exams in four 
subject areas were conducted by a three-year evaluation project 
to obtain information about the effects of a Writing Across the 
Curriculum ("WAC") program in the Minnesota Community College 
System. .This paper will present a brief overview of the project, 
followed by results of the instructional research conducted to 
assess the effects of WAC on the quality of student writing and 
on student mastery of course objectives. 



OVERVIEW OF THE WAC PROGRAM AND WAC EVALUATION PROJECT 

A three-year Writing Across the Curriculum (WAC) staff 
development program was funded by the Bush Foundation to give 
faculty in the Minnesota Community College System the skills and 
motivation necessary to use instructional writing in all subject 
areas. The program involved a four^day summer conference, a one- 
day spring Reunion Conference, plus follow-up meetings and other 
activities organized by a Systemwide WAC Coordinator and the WAC 
Coordinators for each of the 18 community colleges. WAC 
Coordinators attended two leadership training workshops to 
develop the skills necessary to build strong, ongoing college 
programs . 

Program Goals; An accompanying WAC Evaluation Project, also 
funded by the Bush Foundation, was designed to assess the 
effectiveness of WAC in achievin^^ the program's seven goals: 

1 ) To organize WAC workshops and follow-up activities which 
give community college faculty the skills and motivation 
to implement writing activities effectively in the 
courses they teach; 

2) To improve faculty attitudes towards the use of writing 
in instruction; 

3) To increase the amount and types of writing students 
experience across the curriculum; 

4) To obtain diagnostic information which will enable 
faculty to be more effective in their use of writing 
activities; 

5) To improve student attitudes towards writing; 

6) To increase the quality of student writing in courses 
taught by WAC faculty; 

7) To increase student attainment of subject goals in 
courses taught by WAC faculty. 
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Program Evaluation Methodology: A variety of methods requiring a 
high degree of faculty participation was used in collecting 
^[l^?^!!'^^^^" to assess the attainment of program goals. Interview 
scripts, a student essay question and scoring guide, and subject 
experiments were developed, pilot-tested, revised, administered, 
and scored by faculty who were trained to do so. The Evaluation 
Coordinator worked closely with the System WAC Coordinator in 
constructing faculty and student surveys to ensure that they were 
relevant to program needs and interests. All WAC participants 
were asked to complete several faculty surveys. In addition, a 
stratified random sample of 34 professors - some WAC and some 
non-WAC - administered student essays, and 40 other professors 
administered student surveys in one or more of their classes. 
Thus, each of these represents a component of the evaluation 
project which required its own planning, organization, and 
coordination with program personnel. All instruments were pilot- 
tested prior to actual use. 

Instructional Resea rch Components: Assessment of the impact of 
the WAC program on student learning was of greatest interest to 
the sponsoring :igency. The Bush Foundation has been giving money 
for staff development programs to Midwest educational 
institutions on the assumption that the benefits of staff 
development will "trickle down" to students, since faculty 
interviews as well as surveys from students and faculty provided 
evidence that WAC activities sparked enthusiasm (goals #1 and 2) 
and affected classroom practice by resulting in an increase in 
the amount, variety, and perceived effectiveness of instructional 
writing used by WAC faculty (goals §3 and 4), it was possible to 
carry the evaluation project a step further by examining the 
effects of these changes in instruction on students (goals §6 and 
#7). Data were also gathered on student attitudes (goal §5). 

Two types of instructional research studies were conducted as 
part of the evaluation project. 1) Composition Experiment: A 
Systemwide study was implemented to identify the effects of 
instructional writing on the quality of student writing as 
reflected by their performance on a composition question. 
2) Subject Experiments: Experiments in biology, business, 
philosophy, and English were implemented by individual faculty to 
identify the effects of instructional writing on students' 
learning of subject matter as reflected by their performance on 
essay exams in those subjects. Together, these studies provide 
an indication of the potential of WAC programs to achieve their 
instructional goals. A more detailed description of each is 
given below. 



COMPOSITION EXPERIMENT TO ASSESS THE QUALITY OF STUDENT WRITING 



An underlying assumption of the WAC program is that, by having 
students experience more writing and more types of writing in a 
greater variety of disciplines, the quality of student writing 
will improve. Students who have more practice with writing, and 
who are asked to apply principles of good writing in different 
contexts and for different purposes, should be likely to be more 
proficient writers than those whose experiences are restricted to 
composition classes. 

Method; To test this assumption, a 50-minute essay exam was 
administered to 1,085 students in 48 classes across the System. 
The classes were selected from all 18 colleges as a random sample 
stratified by college and subject area, drawn from non-WAC as 
well as WAC faculty. Approximately 400 of these were "baseline" 
essays collected in introductory courses at the beginning of fall 
term, 1987. The others were "post-instruction" essays collected 
from different students - some from the end of fall term, 1987, 
and others from the end of winter term, 1988. The compositions 
were "blind" rated for quality of writing by trained raters using 
a holistic scoring method. For a detailed description of 
holistic rating procedures see White (1985); also White (1989). 

An examination of inter-rater reliability of essay ratings 
indicates that some raters were "easier," and some were "harder" 
than others - that is, they gave ratings consistently above or 
below those given by other raters. However, since each essay was 
scored at least twice by different raters, these individual 
biases balanced each other, and the odds were greatly in favor of 
valid total ratings for each composition. 

Data were analyzed to see whether students who had more 
instructional writing (as indicated by the number of classes they 
completed with WAC-trained professors, plus the number of 
composition classes taken by non-WAC as well as by WAC 
professors) received higher ratings on their compositions than 
students with less instructional writing experience. Additional 
analysis was done to compare post-instruction and baseline essay 
ratings; to see whether a class in which exemplary applications 
of WAC principles were used received higher essay ratings than 
others; and to identify any effects on writing of grammar 
instruction and the use of word processors. 

Results of End-of-Term Versus Baseline Essays; Compos i t i ons 
which were collected at the end of term received significantly 
higher ratings than the baseline essays which were collected at 
the beginning of fall term. The mean essay rating for baseline 
essays was 6.33, and the mean for end-of-term was 6.96. The t- 
test for the difference (898.6 df ) was -4.56, significant at 
p =0. 0001 . The t-test indicates (p=:.0001 ) that the quality of 
student writing improves over a period of one or more terms at a 



community college. One can not be sure that it was instructional 
writing rather than other possible factors (e.g. other types of 
instruction, maturation, motivation from being in a college 
setting over time, etc.) which was responsible for this 
improvement. However, instructional writing was used in most of 
the classes in which essays were collected, and the change 
observed is consistent with the hypothesis that end-of-term 
scores would be higher. 

Results of Cumulative Instructional Writing Experience; Since 
skill in writing may develop more slowly than is observable in 
the period of a lo-week term, historical data were collected to 
identify the impact of instructional writing experienced from the 
time of admission to the community college. A small but positive 
and statistically significant correlation was found between 
students' cumulative writing experience (as reflected by this 
historical measure) and the quality of their writing as measured 
by Lheir composition scores. The historical measure, also known 
as the "WAC'iness" rating, was defined for each student as the 
sum of the number of classes he or she completed from WAC-trained 
professors plus the number of all (including non-WAC) composition 
classes taken. The Pearson correlation coefficient for 1,045 
cases was .066, significant at the .03 level o2 probability. 

No significant correlation was found between writing quality and 
number of ncn-WAC composition courses taken. However, this 
latter variable suffered from low variance, and the absence of 
correlation may be spurious. The total number of classes taken 
overall ("community college experience") was positively 
correlated with essay ratings (the Pearson coefficient was 0.083, 
with p <.007). A partial correlation was calculated to identify 
the effects of WAC'iness ratings when controlled for total 
numbers of classes. The result, r =0.0254, was not significant. 

Results of Exemplary Application Case Study: Hamburg (1979) 
indicates that increasing the amount '-^ writing students do in 
itself will not improve the quality of their writing, and that 
some applications of writing, such as "free writing" for some 
purposes, may even have negative effects. For this reason, a 
Minnesota Community College class was selected to examine the 
impact of instructional writing under hypothesized "best case" 
conditions . 

A professor new to the System designed his history and political 
science classes "from scratch" according to the advice of f ei ^d in 
WAC workshops and materials. The 10-week political science class 
of his which was in the essay collection sample was used as a 
case study to see whether an exemplary application of WAC 
principles would result in higher composition scores. This was a 
regular class, and neither the professor nor the students knew 
that they would be selected for case study analysis until after 
the course had been completed. 
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This exemplar class obtained the highest mean rating of the 48 
involved in the study. A t-test confirmed the difference between 
this and the other classes. The exemplar class mean was 8.24; 
the mean of all other participants was 6.87. The t statistic was 
2.70 (512 df), significant at p =0.0071. These findings suggest 
that the principles advocated by the WAC program are appropriate 
and that instructional writing which is implemented in accordance 
with these principles has the potential to enhance the quality of 
student writing. 

Findings Relating to the Use of Grammar Instruction and of Word 
Processing in the Teaching of Composition; Secondary analysis 
was conducted on data collected for the Systemwide composition 
experiment described above to examine the essay ratings from one 
class in which composition was taught through grammar instruction 
and from four other classes in which composition was taught 
through the use of word processors. Results were consistent wxth 
previous research: 

1) One of the strongest findings in the literature on the 
effectiveness of various instructional procedures in improving 
writing is that grammar study is of little or no benefit 
(Braddock, Lloyd-Jones and Schoer, 1963; Hillocks, 1986). An 
over-emphdsis on grammatical correctness in grading may even 
interfere with the development of writing competence (Adams, 
1971). For this reason, it was hypothesized that a class which 
was taught by a WAC-trained professor but which used grammar 
study instead of instructional writing to teach composition would 
receive lower student essay ratings than average in the 
composition experiment on the quality of student writing. 
Results of a t-test (t= -2.04, df=512, p=0.0416) confirmed the 
hypothesis. 

2) To investigate the effects of word processors on the quality 
of student writing, essay ratings were compared using two classes 
in which word processors were used, versus two regul ir classes. 
In this study, one WAC-trained pjiofessor and one noi. WAC 
professor each had one section of a composition course in which 
students used word processors and one section in which they did 
not. The same final essay exam was given to all four sections 
(two experimental and two control). Essays from the control 
classes were re-typed on a word processor after collection so 
that holistic raters would not know which exams came from which 
classes. In both cases, t-tests revealed that essay ratings for 
the word processing classes were slightly but not significantly 
lower than ratings for the regular classes. (In the first study, 
t=0.28, df=43, and p=0.78; in the second, t=1.18, df=42, and 
p=0.24). These results reinforce conclusions reached in most 
earlier studies that student writing does not improve with the 
use of a word processor (McAllister and Louth, 1988; Harris, 
1987; Hawisher, 1986). 
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SUBJECT EXPERIMENTS TO ASSESS STUDENT MASTERY OF COURSE 
OBJECTIVES 

Another assumption of WAC is that instructional writing will help 
students to process information, thereby improving their 
understanding and retention of course content (Fulwiler, 1985; 
also Myers, 1985). Survey results from the evaluation project 
indicate that over 90% of both faculty and students believe that 
instructional writing assists the learning of subject matter. 
Professors' interest in WAC may be based on this bilief. 

Method: Six quasi -experimental studies in four subjects - 
biology, business, language arts and philosophy - were conducted 
to test the effects of instructional writing on student 
attainment of subject objectives. The experiments were conducted 
by faculty who received training in designing and implementing 
the experiments. Instructional writing was used in the 
experimental sections of these courses, and alternative 
instruction was used in control sections. Experimental and 
control classes were comparable in their course content and 
student characteristics. 

An essay exam, business letter, or some other exam involving 
subjective judgment was collected from students in both sections 
in five of six of the experiments - the sixth used an objective 
test. The subjective exams were "blind rated" by additional 
faculty from the same four subject areas who were trained as 
trait raters and who constructed scoring guides which addressed 
specific course objectives. Data from each experiment were 
analyzed to see whether the students in the instructional writing 
sections achieved higher ratings on their subject essay or 
objective exam than students who received another type of 
instruction , 

Findings: In five of the six studies, the experimental class 
obtained slightly higher mean scores on their exams than did the 
control class. (See Table 1: "Results of WAC Subject 
Experiments.") Since t-tests indicate that none of the 
differences in means was statistically significant, these studies 
when considered individually do not provide convincing evidence 
that writing improves student learning of subject matter. 
However, outcomes of the experiments together constitute what may 
be an emergent pattern. The probability of seeing five pro- 
experimental differences in six trials by chance alone is 11%. 
More subject experiments are needed to see whether other outcomes 
reinforce or negate this pattern of results. 
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STUDENT ATTITUDES TOWARDS WRITING 

Analysis of end-of-project student survey data using the Chi- 
Square statistic indicates that students who have had more 
writing experience in their community college also have more 
positive attitudes towards writing (see Tables 2-5). Those 
student respondents who reported having a lot of writing in most 
of their classes agreed or strongly agreed that writing helps 
them to learn subject material. In addition, significantly 
larger percentages of those who have had more instructional 
writing agree that they enjoy writing or that it gives them a 
sense of satisfaction; that writing assignments are preparing 
them for their chosen occupation; and that writing is important 
in their personal life. 

An unexpected result of our analysis was the discovery that 
students have more positive attitudes towards writing than 
faculty seem to think. As shown in Table 6 ("Student Attitudes 
Towards Writing"), students give higher ratings to their own 
enjoyment of writing than faculty give in assessing their 
students' enjoyment of writing. 



ClSCUSSION AND IMPLICATIONS FOR FURTHER STUDY 

Instructional research designed to identify the effects of 
instructional writing on student writing proficiency and subject 
learning were proraising in both areas. However, further research 
is needed to substantiate these findings. The pattern of results 
observed in the subject experiments was suggestive, though not 
statistically significant, and will therefore require additional 
studies for confirmation. Implications of the composition 
experiment need clarification, since although they were of 
statistical significance, the results were tenuous and somewhat 
ambiguous. 

Possible Faculty Variables: Composition experiment correlations 
were so small that they may be explained by faculty variables. 
Since professors who participate in the WAC program do so on a 
voluntary basis, it may be that they are "better teachers" than 
non-WAC professors. If so, it may be that it is better teaching 
in a general sense, rather than the specific use of instructional 
writing, which has a positive effect on the quality of student 
essays . 

Total Number of Class Even if the composition results are not 
due to better teaching by WAC faculty, the impact of 
instructional writing as distinct from total number of classes 
taken remains unclear. It is possible that it is not 
instructional writing, but rather some other aspect or aspects of 
classroom experience that resu3.cs in improved student writing. 

7 

ERiC S 



stimulation of additional classes are of greater importance. On 
the other hand, it may be that instructional writing is, indeed, 
the key factor, but that the effects of the WAC'iness ratings are 
clouded because students experience a substantial amount of 
writing in classes taught by non-WAC faculty. If so, WAC'iness 
ratings may be too imprecise to serve as a good measure of 
writing experience. 

Student Variables; Another possibility is that students who take 
more courses are better motivated and have greater academic- 
aptitude than those who complete fewer courses. If so, such 
student characteristics could account for the correlation between 
essay ratings, number of WAC classes and total number of classes. 

Instruct ional Design: Given the relative clarity of results for 
the exemplary application case study and findings from previous 
research, it is perhaps most likely that instructional writing 
can be expected to be effective only when applied according to 
certain principles and under certain conditions - that whether 
writing is used is not as important as the ways in which it is 
used for subject learning as well as for writing proficiency. 
Studies suggest that some kinds of writing instruction may have a 
negative effect on student compositions (Hamburg, 1979); and that 
although notetaking and summary writing can promote subject 
learning (Kulhavy, Dyer, and Silver, 1975; Spurlin, Dansereau, 
Larson, and Brooks, 1984), writing which requires analysis and 
evaluation is more beneficial (Newall, 1984; Durst, 1987). Thus, 
for subject experiments as well as for the composition 
experiment, a -otential for positive results may be diminished 
through the . .e amount and types of writing used by different 
professors, and from an unevenness in the qualit ' of the design 
and implementation of instructional writing activities. 

Further Research; Another three-year grant was received by the 
Minnesota Comraunity College System from the Bush Foundation to 
institutionalize the programs and continue the evaluation 
project. During this period, secondary analysis will be 
performed to identify the effects of the competing variables 
discussed above. A more intensive longitudinal composition 
experiment will be conducted in three Greater Minnesota colleges, 
in which pre-test and post-test essays will be collected from the 
same students. Additional subject experiments are also 
anticipated. 
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CONCLUSION 

In combination, findings from the WAC evaluation project provide 
evidence that, if instructional activities are well-designed, 
more experience with writing in courses across the curriculum can 
help students to become better writers. More research is needed 
to determine whether instrvctional writing, per se, is effective, 
and what intervening variables might enhance or impede outcomes. 
Principles of good instructional design promulgated by the WAC 
program are supported by our results, but need further 
exploration. Outcomes of individual classroom experiments on the 
effects of instructional writing on the learning of subject 
matter are promising but are not statistically significant. 
Additional experiments must be conducted to confirm or deny the 
existence of an emergent pattern of gains which may not register 
statistical significance du ^ to the small numbers of students and 
short period of treatment (i:en weeks) involved. 
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TABLE 1 

RESULTS OF WAC SUBJECT EXPERIMENTS 



"P" VALUE 

EXPERIMENTAL CONTROL t-STAT- (LEVEL OF 

EXiERIMENTS GROUP MEAN GROUP MEAN ISTIC df SIGNIFICANCE) 

Biology 1 7.68 6.62 -i.30 54 .20 

Biology 2 10.54 9.37 -.79 25 .44 

English 8.75 7.65 -1.49 27 .15 

Philosophy 25.76 25.14 -0.37 41 .72 

Business 1 7.22 6.42 -0.92 32.3 .37 

Business 2 6.90 7.29 .68 59 .50 
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107 91 


940 
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TABLE 5 



51U0ENT 6XP6«^ieNC6 ANO ATTlTUOeS TOWARD WillTING* (R6C0060I 

WRirii^G IS IHPORTANT for P6F50NAL LIF6< 



SlQl 
EXPECTED 

ROW per 

COL PCT 



TABLE OF SlQl BY S6Q13 
WRITIHC 00N6 IN CC CLASSES. S6C13 



NO/vGR^ Li rr.E 



FAIR AMOUNT OF W 



GREAT DEAL OP WR 



LOTS OF WRITJfiC 



(AGREE 
I 



0 I 
• I 



16 

28.07 
10.74 

80 
78.7 

93. 6<' 



39 
35.8 
^7.56 
26.17 



9.6 
63.64 
9.40 



UHCERTAllOtSAGREE 
N I 



I 


1 


I 


. 


1 


. 




1 






1 




IT 


1 


24 



16.9 I 

29.82 I 

16«83 I 

56 I 

53.3 I 

31. II I 

55»45 I 



15.2 
42.11 
26.37 

44 
48.0 
24.44 
48.35 



21 I 

24.3 I 

25.61 I 

20.79 I 



22 
21.9 
26.8 3 
2^.18 



7 I 

6.5 I 

31.82 i 

6»93 I 



I 

5.9 

4.55 
1. 10 



TOTAL 



TOTAL 



149 



101 



91 



57 



180 



82 



22 



3^1 



u !t oO > E 
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TABLE 6 

STUDENT ATTITUDES TOWARDS WRITING 



WAC Faculty Perception Student 

of Student Attitudes* Self -Ratings** 

§ % (if % 

ENJOY WRITING 9 8.2 233 23.5 

SENSE OF ACCOMP- 54 49.1 318 32.0 

LISHMENT 

DON'T MIND WRITING 37 33.6 299 30.1 

ALMOST NEVER LIKE 7 6.4 110 11.1 

DON'T USE INSTRUC- 3 2.7 

TIONAL WRITING 

AVOID TAKING 33 3.3 

CLASSES WITH WRITING 

100.0 100.0 



*The question WAC-trained faculty were responding to was, "Which 
statement best describes your students' attitude towards 
writing?" 

**The corresponding question for students was, "which statement 
best describes your attitude towards writing?" 




TABLE 7 



RESULTS OF WAC EXPERIMENT RELATING TO THE USE 
OF GRAMMAR INSTRUCTION IN THE TEACHING OF COMPOSITION 



"P" VALUE 

MEAN OF ESSAYS MEAN OF ALL t-STAT- (LEVEL OF 

FROM GRAMMAR CLASS STUDENT ESSAYS ISTIC df SIGNIFICANCE) 

5.82 6.97 -2.04 512 0.04 



TABLE 8 



RESULTS OF WAC EXPERIMENTS RELATING TO THE USE 
OF WORD PROCESSORS IN THE TEACHING OF COMPOSITION 



MEAN OF ESSAYS 
FROM WORD PRO- 
CESSING CLASS 



MEAN OF ESSAYS 
FROM REGULAR t-STAT- 
CLASS ISTIC 



Experiment 1: 6.81 



df 



7.00 



- 0.28 



43 



"P" VALUE 
(LEVEL OF 
SIGNIFICANCE) 



0.78 



Experiment 2: 6.78 



7.57 



-1.18 



42 



0.24 
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